Silk: A Resilient Routing Fabric for Peer-to-Peer Networks

Several proposed peer-to-peer networks use hypercube routing for scalability. In a previous paper, we showed thatconsistency of neighbor tables in hypercube routing guarantees the existence of a path from any source node to any destination node. Consistency, however, can be broken by the fail-ure of one node. To improve the robustness of hypercube rout-ing, we generalize the concept of consistency to K-consistencyfor K 1. We then show that a K-consistent hypercuberouting network provides at least K disjoint paths from anysource node to any destination node with a probability close to1. The first objective of this report is the design and specification of a new join protocol together with a proof that it gener-ates K-consistent neighbor tables for an arbitrary number ofconcurrent joins (under the assumption that there is no concurrent leave or failure). To do so, we construct a more generaldefinition of C-set tree than our previous one as the concep-tual foundation for protocol design and reasoning about K-consistency. Both the new protocol and proof require majorextensions to the ones in our previous paper to generalize themfrom 1-consistency to K-consistency. The second objective of this report is the design and evalu-ation of a failure recovery protocol for K-consistent networks.From simulation experiments in which up to 50% of the nodesin aK-consistent network failed (when a node fails, it becomessilent), we found that, for K 2, K-consistency was recov-ered in every experiment. The third objective of this report isto extend our join and failure recovery protocols such that theyconstruct and maintain K-consistent neighbor tables for net-works whose nodes join and fail concurrently and frequently.In particular, our join protocol is extended with rules to handle failures of not only existing nodes but also other joining nodes. These extended protocols, being implemented in ourprototype system named Silk, will be referred to as Silk protocols. From simulation experiments in which the number ofconcurrent joins and failures was up to 50% of the initial net-work size, we found that, for K 2, Silk generated and mainResearch sponsored by NSF grant no. ANI-9977267 and Texas AdvancedResearch Program grant no. 003658-0439-2001.tained K-consistent neighbor tables after the concurrent joins and failures in every experiment. We also present an analysisof the communication and storage overheads of Silk protocols and show that Silk is scalable to a large number of networknodes.